Guest Editorial: Challenges and Perspectives for Affective Analysis in Multimedia

THERE has been an increasing interest in the research on multimedia indexing and retrieval based on subjective concepts, such as emotion, preference and aesthetics. The research efforts in this direction are considered human-centered and beyond the conventional keywordor (semantic) object-based multimedia analysis paradigm [1]. In addition, the problems addressed by these efforts are considered challenging since they require multidisciplinary understanding of human behavior and perception as well as multimodal approaches integrating different modalities, e.g., audio, visual and text, to reach an acceptable level of performance. In addition to the multimedia content and to better take into account the human factors, users’ spontaneous responses, either explicit (e.g., rating) or implicit (laughing, gaze direction, physiological responses) are increasingly also integrated as input in such approaches. This special section focuses on affective analysis of multimedia, that is, the analysis focusing on estimating the expected emotional state of the user when interacting with multimedia content. The result of such analysis can enhance multimedia retrieval and recommendation systems by adding the emotion-related factor into the assessment of the appropriateness of the content for the user [2]. The mission of the special section is to provide a view at the state-of-the-art in this domain and point to the potentially interesting future research directions. It therefore features articles concerning multimedia affective understanding from different perspectives, including predicting likability and emotions from users’ responses [3], [4], content analysis for music affective understanding [5], [6], [7], visual content analysis for aesthetics and emotional understanding [8], [9], [10], and an application for sentiment analysis [11]. In response to an open call for our special section, we initially received 35 submissions [4], [10] from which we accepted nine original articles. In the next section, we summarize the articles and their key findings. In the final section, we summarize the challenges for future research on affective analysis in multimedia.